1. Field of the Invention
The present invention relates to an image decoding device for performing, in decoding, coding data which include bidirectional predictive coding data.
2. Related Art Statement
In recent years, in keeping with establishment of a high-efficiency coding technique of an image, digital processing of the image has been diffused. The high-efficiency coding technique is such that image data are performed in coding by a less bit rate, in order to improve efficiency of digital transmission, recording and the like. In this high-efficiency coding, orthogonal transformation such as DCT (discrete cosine transformation) processing or the like is performed by a block unit of m.times.n pixels. The orthogonal transformation is one which transforms an inputted sample value to an orthogonal component such as a spacial frequency component or the like. Thus, a spacial correlation component is so made possible as to be deleted. The component which is transformed in orthogonalization is quantized whereby redundancy of the signal of the block is reduced.
Moreover, variable-length coding such as Huffman coding or the like is applied to the quantization output whereby an amount of data can further be reduced. The Huffman coding is one in which coding is performed on the basis of the results which are computed from the amount of statistical codes of the quantization output. A whole amount of data is reduced by the variable-length coding in which short bits are allocated to data which are high in appearance probability, and long bits are allocated to data which are low in appearance probability.
In a device for performing the high-efficiency coding, a hybrid system which is employed in MPEG (Moving Picture experts group) or the like has become the main current. In this system, inter-frame compression in which inter-frame correlation is utilized to curtail or reduce redundancy in a direction of a time axis is also employed, in addition to intra-frame compression in which an intra-frame image is processed in DCT. The inter-frame compression is arranged such that such a property that general animations well resemble each other in before and after or before and behind frames is utilized to find out a difference between the before and behind frames, and a difference value (predict error) is performed in coding, whereby a bit rate is further reduced. Particularly, motion-compensation inter-frame predictive coding is effective in which motion of the image is predicted to find out an inter-frame difference, to thereby reduce the predictive error.
In this manner, in the hybrid system, in addition to the intra-frame coding in which the image data of the predetermined frame are processed in DCT as they are and are coded, the predictive coding is employed in which only the difference data between the image data of the predetermined frame and the reference image data of the frames before and behind this frame are processed in DCT and are performed in coding. A predictive coding method includes forward predictive coding in which the reference image data in a direction which is front or forward in view of time are compensated in motion to find out a predictive error, rearward predictive coding in which the reference image data in a direction which is rearward in view of time are compensated in motion to find out a predictive error, and bidirectional predictive coding which uses the average in either one direction of the forward and the rearward or in both directions, in consideration with the coding efficiency.
Since the frame which is performed in coding by the intra-frame coding (hereinafter referred to as an "I picture) is performed in coding only by the intra-frame information, the frame is capable of being decoded only by independent coding data. Accordingly, the arrangement is such that, in an MPEG standard, one I picture is inserted into a fixed period or a fixed cycle (12 frames, for example) in order for prevention of error propagation, or the like. In the MPEG standard, the inter-frame coding frame (hereinafter referred to as a "P picture") is obtained by the forward predictive coding which uses this I picture. In this connection, the P picture is also obtained by the fact that the forward P picture is performed in coding in forward prediction. Moreover, the bidirectional predictive adaptation change-over frame (hereinafter referred to as a "B picture) is obtained by the bidirectional predictive coding which uses the I and P pictures in either one direction or in both directions of the forward and the rearward.
FIG. 1 is a block diagram showing the related art of the image decoding device for performing in decoding the coding data which have such I, P and B pictures.
The coding data which are inputted through an input terminal 1 are supplied to a stream decoding circuit 2. These coding data are ones to which the above-described high efficiency coding is applied, and are obtained by the fact that the image data or the predictive error are/is processed in DCT and are/is quantized, thereafter, are/is performed in coding in variable length. The stream decoding circuit 2 performs in decoding in variable length the inputted coding data and, thereafter, the inputted coding data are quantized in reverse so as to be outputted to an inverse DCT circuit (hereinafter, referred to as an "IDCT circuit") 3. The IDCT circuit 3 processes in inverse DCT the inverse quantization output to return the same to data prior to DCT processing on the side of coding.
The data which are performed in decoding by the IDCT circuit 3 are transmitted to a memory data bus 5 through a motion compensation circuit (hereinafter, referred to as an "MC circuit") 4. An address generator unit (hereinafter, referred to as an "AGU") 6 transmits the decoding data which are transmitted through the memory data bus 5, to a memory 7 to store the same in the memory 7.
Now, it is assumed that the coding data of an I picture are inputted. In this case, an output from the IDCT circuit 3 is a restored image of the frame. The output from the IDCT circuit 3 is given to the memory 7, as it is, through the MC circuit 4 and is stored. The output from the IDCT circuit 3 is pixel data in a block unit. The memory 7 holds the pixel data for a 1 frame.
Next, it is assumed that a P picture is performed in decoding. In this case, the output from the IDCT circuit 3 is a predictive error. The MC circuit 4 reads the restored image data of the I picture that is a reference image from the memory 7, to compensate, in motion, the same by the use of the motion vector. The MC circuit 4 obtains the decoding data of the P picture by addition between the restored image data of the I picture which are compensated in motion, and the predictive error. The decoding data of the P picture from the MC circuit 4 are transmitted through the memory data bus 5, and are written to the memory 7 by the AGU 6.
Next, it is assumed that a B picture is performed in decoding. Also in this case, the output from the IDCT circuit 3 is a predictive error. The MC circuit 4 reads the restored image data of the reference image which is stored in the memory 7, and uses the motion vector to compensate, in motion, the same to thereby add the same to the predictive error. Thus, the MC circuit 4 obtains the restored image data of the B picture. These image data are given to the memory 7. Thereafter, similarly, decoding is performed. The AGU 6 reads the data which are stored in the memory 7, in display order, and outputs the same through a display processing circuit 8.
By the way, the memory 7 has a region which stores the reference image (hereinafter, referred to as a "reference-image memory region"), and has a memory region for transforming the restored image in interlace order (hereinafter, referred to as a "display transformation memory region"). Specifically, when the decoding data are of a progressive (frame) arrangement, in order to perform interlace display, it is necessary to perform transformation from progression to interlace (hereinafter, referred to as an "interlace transformation").
The P picture is performed in decoding by the use of the reference image of the forward frame. Since the P picture holds the reference image for the decoding, the reference-image memory region for the 1 frame is necessary. Further, the B picture is performed in decoding by the use of the reference images of the forward and rearward frame, and the reference-image memory region for the 2 frames is necessary for holding these reference images. Moreover, since the coding processing is performed in the DCT block unit, the display transformation memory region for the 1 frame is necessary for framing the coding data from the MC circuit 4 and, thereafter, for outputting the same in the interlace order. In this case, the decoding data of the I and P pictures are stored in the reference-image memory region of the memory 7 in order to be used as the reference image of the B picture. Read from this region is controlled and is outputted, whereby this region can be used both as the display transformation memory region for interlace transformation. However, since the decoding data of the B picture are not used as ones for the reference image and are not stored in the reference-image memory region, the display transformation memory region for the interlace transformation is necessary. Specifically, since only the decoding data of the B picture should be stored in the display transformation memory region, this region will hereinafter be referred also to as a "B picture region".
Next, write and read from and to the display transformation memory region when the interlace transformation is performed will be described with reference to FIG. 2. In FIG. 2, a horizontal axis indicates time of the frame unit, while a vertical axis indicates an address space in the display transformation memory region. Furthermore, oblique line parts and oblique lines in a left side of screening parts indicate write of the decoding data. Oblique line parts and oblique lines on a right side of the screening parts indicate read of the coding data, while the oblique line parts and the screening parts indicate that the decoding data are held. Further, the oblique line parts and the thick line on the right side of the screening parts indicate read at the time of enlarged or magnified display. In this connection, the capacity of the display transformation memory region is assumed to be for a 1 frame, and the capacity of each of the regions A to D is either 1/4 the full capacity.
The display transformation memory region of the memory 7 has four regions including A to D. In order to use the memory for a 1 frame to interlace the image data of a 1 frame to thereby output the same, it is necessary to perform the write and the read simultaneously. In view of this, the arrangement is such that the image data of the 1 frame are divided into four, and four regions A to D are provided, to control the write and the read every regions.
Specifically, it is assumed that the image data of the first field are divided vertically of the image plane so as to be made to data #1 upper and data #1 lower, respectively. Furthermore, the image data of the second field are also divided vertically of the image plane so as to be made to data #2 upper and data #2 lower, respectively. The data #1 upper and the data #2 lower are stored respectively in the regions A and D, and the data #1 lower and the data #2 upper are alternately stored respectively in the regions C and D.
When the coding data which are performed in coding by the frame arrangement are performed in decoding, the decoding output from the MC circuit 4 is successively outputted in the block unit from the first block line corresponding to the upper end of the image plane, to the block line corresponding to the lower end of the image plane. Specifically, in the image decoding device in FIG. 1, the data #1 upper and the data #2 upper are performed in decoding in the 1 field period the first half the decoding frame, and the data #1 lower and the data #2 lower are performed in decoding in the next 1 field period.
In the first 1 field period of time, the data #1 upper and the data #2 upper which are performed in decoding are respectively written successively to the regions A and C. Oblique lines K1 and K3 in FIG. 2 indicate write to the regions A and C. The amount of data of the data #1 upper and the data #2 upper is 1/4 the 1 frame, and is written to all the area of the regions A and C in the 1 field period of time.
In the next 1 field period of time, the data #1 lower and the data #2 lower which are performed in decoding are respectively written successively to the memories B and D. Oblique lines K2 and K4 in FIG. 2 indicate write to the regions B and D. The amount of data of the data #1 lower and the data #2 lower is 1/4 the 1 frame, and is written to all the area of the regions B and D in the 1 field period of time.
Further, in this field period of time, read is performed from the region A. The oblique line R1 in FIG. 2 indicates the read from the region A. The data are read from the region A in write order so that all the data #1 upper which are stored in the region A are read in the half field period of time. Moreover, by the oblique line R2 in FIG. 2, it is indicated that the data #1 lower which are stored in the region B are also read, in the latter half of this field period of time. The data #1 upper and the data #1 lower which are read in this field period of time are outputted as data of the first field of the display frame.
In the first half of the next field period of time, the data #2 upper which are written to the region C in the period of the oblique line K3 are read (oblique line R3). In the latter half, the data #2 lower which are written to the region D in the period of the oblique line K4 are read (oblique line R4). Thus, the data #2 upper and the data #2 lower are outputted as data of the second field.
Subsequently, similar operation is repeated, and the regions A to D are suitably assigned such that the write regions are not overlapped with each other, whereby it is possible to obtain the interlace output successively.
By the way, the read from the memory 7 is controlled, and predetermined filtering processing is applied to the decoding data, whereby it is also possible to magnify or enlarge and display a part of the image, for example, in a vertical direction.
FIG. 3 is an explanatory view for describing interpolation processing of the display processing circuit 8 when a system for performing the interpolation processing in division into an image of a first field and an image of a second field is employed (hereinafter referred to as a "field interpolation system"). An example in FIG. 3 shows the interpolation processing for display a lower half of the image in magnification or enlargement twice in the vertical direction.
FIG. 3 shows the read of the memory 7 in the first field and the second field. Marks .largecircle. show decoding data which are read from the memory 7, while marks x show interpolated data.
In the first field, the display processing circuit 8 uses the restored pixel data of the odd line (marks .largecircle.) which are read from the memory 7, to interpolate the pixel data of the even line (marks x). Since it is magnified twice vertically, a 1 interpolation line should be created between the respective read lines. That is, the display processing circuit 8 increases 1/2 times the sum of the odd lines upper and lower the interpolated even line, to thereby obtain the interpolation data of the even lines.
Similarly, in the second field, the display processing circuit 8 increases 1/2 times the sum of the pixel data of the adjacent two even lines which are read from the memory 7, to thereby obtain the interpolation data of the odd lines which are shown by the marks x. Thus, the image of the lower half shown in the oblique line parts in FIG. 4A is magnified to an image which is shown in oblique line parts in FIG. 4B.
In this connection, the read of the memory 7 in this case is shown by thick lines on the right side of the oblique line parts and the screening parts in FIG. 2. Further, setting information such as a display region of a magnified image is so arranged as to be supplied from a host interface (hereinafter referred to as an "HIF") 9 to the display processing circuit 8 through a setting data bus 10.
Moreover, FIG. 5 is an explanatory view for describing the interpolation processing of the display processing circuit 8 when a system for performing the interpolation processing (hereinafter referred to as a "frame interpolation system") is employed with respect to an image which has the frame arrangement. An example in FIG. 5 also shows the interpolation processing for displaying a lower half the image in magnification twice in the vertical direction. Also in FIG. 5, marks .largecircle. show decoding data which are read from the memory 7, while marks x show interpolated data.
In the first field, a portion between the first and third lines is interpolated by the use of the pixel data of the second line of the second field. Further, in the second field, a portion between the second and fourth lines is interpolated by the use of the pixel data of the third line of the first field.
When the coding data of a progressive image like motion pictures are inputted to the input terminal 1 in FIG. 1, the image quality is improved in one which employs the frame interpolation system. As will be clear from the above description, however, in the frame interpolation system, the pixel data of the first field and the second field are necessary also in either of the first field period of time and the second field period of time.
Specifically, if the frame interpolation system is adopted or employed, the data #1 lower which are stored in a B picture region of the memory 7, for example, must continue to be stored in the B picture region until the data #2 lower are used in display. For this period, the region B or C of the B picture region is not opened so that the next data #2 upper cannot be written. Accordingly, when the memory capacity for the 1 frame is set as the B picture region, it is impossible to employ the frame interpolation system because of insufficiency of the memory capacity.
In this manner, there are the following problems. That is, when a part of the image is displayed in magnification, for example, it is impossible to perform the interpolation processing with respect to the image which has the frame arrangement, because of insufficiency of the memory capacity, in spite of the size of the displayed image. Accordingly, it is impossible to improve the image quality.